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IMAGE TRANSFORM METHOD FOR OBTAINING EXPANDED IMAGE DATA, 
IMAGE PROCESSING APPARATUS AND IMAGE DISPLAY DEVICE THEREFOR 

Field of the Invention 

The present invention relates to an image transform method 
5 for processing original input image data, and in particular, 
to an image transform method for reducing the step-shapes or 
chain-shapes of oblique lines, which appear when original 
input image data are expanded, and for controlling the 
;fi distortion of constituent lines, and to an image processing 
CttO apparatus and an image display device therefor, 

S Background Art 

HI Conventionally, the nearest neighbor method and bilinear 

hi interpolation have been used widely as interpolation methods 

D to obtain a match between a high definition liquid crystal 

: 15 display (LCD) and a lower resolution personal computer (PC) 
screen. These interpolation methods are popular because 
only a small number of calculations is required for the real 
time processing of a high resolution image. However, it is 
known that the nearest neighbor method contributes to an 
20 increase in distortion in an expanded image, and therefore, 
the bilinear interpolation method, whereby the rough shape 
of an original image can be expanded, is often employed. 
But it is also true that bilinear interpolation tends to 
blur resultant images, an effect that is especially 
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remarkable when bilinear interpolation is employed for 
images displayed on an image display device, such as an LCD, 
that does not have the low-pass characteristics. 

The bilinear interpolation method is an example of the 
5 multirate processing theory called linear interpolation. 
And because of this, interpolation methods based on the 
general multirate processing theory have been proposed that 
use more pixels than the number employed for bilinear 
interpolation. For example, a multirate interpolation 
£10 method using a table search method for a linear filter is 
X [ disclosed in USP 5,410,616. Further, in USP 5,594,676, 

0 linear filters having different orders are prepared, and are 

il changed in accordance with the characteristics exhibited by 

y peripheral pixels. In addition, according to reference 

LJ5 document 1 (IEEE Signal Processing Magazine, Vol. 16, No. 6, 
11 pp. 22-38, November 1999), pertaining to the generalization 
Z of the interpolation method, except for a point for 

□ obtaining an efficient number of calculations, a linear 

interpolation method whereby information loss can 
20 theoretically be minimized can be substantially provided as 
an interpolation method that uses a third-order B-spline. 
Thus, it becomes apparent that image processing using the 
linear interpolation method is approaching the limit of its 
effectiveness . 

25 When the above method is employed for image interpolation, 
vertical calculations and horizontal calculations are 
sequentially performed, and one pixel in a low-resolution 
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image is transformed into a set of rectangular pixel 
regions, so that the appearance of step-shapes or 
chain-shapes of oblique lines can not be prevented. In 
order to prevent the appearance of such shapes, it is 
5 considered effective to perform interpolation by employing a 
directional filter that uses peripheral pixels aligned in 
the direction in which an oblique line is extended. For 
example, in reference document 2 (IEEE Proceedings of the 
1995 International Conference on Acoustics, Speech, and 
10 Signal Processing, Vol. 4, pp. 2383-2386, May 1995), and in 
USP 5,991,463, the use of the directional, linear 
^ interpolation methods are proposed. 

!;| These image expansion methods are required for a computer 

:fj monitor and a projector in order to provide a match between 
LL5 a flat panel display, such as an LCD, which has a fixed 
ijl pixel resolution, and an image produced by the graphics card 

!r! of a PC. However, when an expansion process for an original 
Q image is performed vertically and horizontally, or in the 

r " opposite order, as with the conventional bilinear 

20 interpolation method, correlation is increased vertically 

and horizontally for the pixels that have been interpolated. 

As a result, when an oblique line is expanded, a pixel 
displayed on a low-resolution screen is transformed into a 
group of rectangular pixel regions, and a step-shape or 
25 chain-shape of oblique line appears. And even when a high 
accurate linear interpolation is performed using a third 
order B-spline, the above phenomenon becomes ever more 
remarkable as the expansion ratio is increased. When this 
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phenomenon appears on the display of a high definition LCD, 
it seems to the user that the physical definition of the 
output screen differs from the actual definition. 



In reference document 2, the differences in pixels are 
5 calculated for predetermined directions, and the reciprocals 
of values obtained for all the directions are added together 
to acquire the normalized weight coefficient. Linear 
interpolation is performed for pixels dispersed in each 
direction corresponding to each weight coefficient, and to 
JlO provide an expanded image, the summation of the results is 
^ calculated based on the weight coefficients. In this 
:n method, linear interpolation is performed by regarding, as 
;f; the dominant direction, the one in which the pixel 

rfl difference is small, and this method can be considered as a 

y5 method for preventing the step-shapes or chain-shapes of 
jf| oblique lines. However, using this method, since in order 
:~ to prevent an erroneous determination the summation is 

O calculated by also using the interpolation results obtained 
5 ~~ for the non-dominant direction, blurring of an output image 
20 can not be avoided. 

In USP 5,991,463, a bilinear interpolation technique using 
four adjacent pixels arranged in a specific direction is 
proposed as a directional interpolation method. 
Specifically, according to this method, first, a difference 
25 among four peripheral pixels is obtained for the 

determination of three directions, i.e., the right oblique, 
the left oblique and the vertical directions. The 
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differences are compared with each other, and the direction 
having the smallest pixel difference is determined to be an 
interpolation direction. In addition, an additional search 
is performed, as needed. When the pixel differences for the 
5 three directions are substantially the same, the vertical 

direction is determined. When the pixel difference for the 
vertical direction is small and the difference for one of 
the oblique directions is obviously small, the pertinent 
oblique direction is determined. When the pixel differences 
.JLO for the two oblique directions are small, the same direction 
5 determination is performed for a pixel that forms an oblique 
ffl line parallel to the pixels whose difference was calculated. 

=fj And when the differences for the two oblique directions are 

Jfj small for the additional pixel, the vertical direction is 
:£15 determined. In this manner, as is disclosed in this 
publication, when interpolation is performed for one 
ill specific direction, excessive image blurring can be avoided. 

D However, according to the above method, an erroneous 

determination may be made if in an observation pixel mask 

20 used for direction determination foreground and background 
can not be identified, such as when white and black pixels 
intersect to form an X shape. When the direction can not be 
correctly determined, an interpolation value is obtained 
that is far different from the correct value, and there is a 

25 defect in the interpolation results. In addition, 

generally, according to the expansion method for which only 
oblique interpolation is performed, correlation in the 
vertical and horizontal directions is reduced. That is, 
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since at the point where vertical and horizontal linear 
lines intersect, an image is expanded as if it is assumed as 
a diamond shape, instead of expanding in the original 
vertical and horizontal directions, the obtained image is 
5 distorted. Further, since the isolated point of the 

original image is determined to be the point whereat the 
oblique lines cross, and the image is to be expanded in the 
oblique directions, the isolated point is faded out, or the 
expansion of the image is less than it is supposed to be. 

iffO To resolve the above shortcomings, it is one object of the 

present invention to provide an image transform method 
=0 whereby the step-shapes or chain-shapes of oblique lines are 

;J; not remarkable, even when an image has been expanded by two 

=n or more times of its original size. 

LI 5 It is another object of the present invention to output a 
01 clear, expanded image wherein distortion is eliminated and 

no constituent lines of fonts and graphics are missing. 
O It is an additional object of the present invention to 
" perform fast image transform processing. 

20 Summary of the Invention 

To achieve the above objects, according to the present 
invention, based on a linearly expanded image that is 
interpolated in the vertical and horizontal directions, or 
vice versa, an intermediate process image is obtained 
25 wherein the vertical and horizontal correlation is reduced. 
Further, the direction correlated with the target pixel is 



DOCKET NUMBER: JP920000126US1 



detected, and the image is expanded by using neighboring 
pixels arranged in the detected direction to produce a 
resulting image. The processing of the intermediate image 
and the resulting image are employed to generate a final, 
5 expanded image. Specifically, according to a first aspect 
of the present invention, an image transform method, for 
transforming original input image data into image data 
expanded by a ratio represented by a rational number or an 
integer, comprises the steps of: reducing correlation in the 
10 vertical and horizontal directions of a linearly expanded 
fi image to generate first expanded image data; performing 

3 linear interpolation, based on correlation with a target 

fl pixel constituting the original image data and neighboring 
i\ pixels arranged in oblique directions, using the neighboring 
fi.5 pixels to generate second expanded image data; and employing 
L the first expanded image data and the second expanded image 

fi data to generate a final image. 

:□ The step of generating the first expanded image data 

includes the steps of: raster-scanning a window having a 
20 predetermined size wherein the target pixel and the 

neighboring pixels in the linearly expanded image data are 
included; and reducing vertical and horizontal directional 
correlation through a rank order processing using median 
operation in the window. This configuration is preferable 
25 because to perform the process for reducing the vertical and 
horizontal directional correlations, simple operations can 
be used for which no enormous amount of calculation is 
required. 
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Further, the step of generating the second expanded image 
data includes the steps of: determining an interpolation 
direction based on values of differences between the target 
pixel and the neighboring pixels; and performing linear 
5 interpolation in the determined direction. This step leads 
to relatively precise determination of interpolation 
direction. 

The image transform method further comprises the step of: 
5 regarding, as an adjustment value, the personal preference 
Hlo of a user concerning image quality, wherein, at the step of 
=;0 generating the final image, based on the adjustment value, 

If* the final image is generated by using the first and the 
gg second expanded image data. As a result, an output image 

L. can be obtained by extracting suitable portions from the two 

o!5 sets of expanded image data, e.g., by reducing a bulging 
:™ shape that is exaggerated by the second expanded image data 

D and appears at a crossing point. 

According to a second aspect of the invention, an image 
transform method comprises the steps of: forming an image by 

20 linearly expanding original image data in the vertical and 
horizontal directions; and reducing the vertical and 
horizontal directional correlation of the image through a 
rank order processing using, for example, median operation 
to generate an expanded image . 

25 The image transform method further comprises the steps of: 

determining, for the expanded image, whether the contrast in 
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the original image data can be maintained at a predetermined 
level; and extracting a high frequency component from the 
expanded image, when the contrast can not be maintained at 
the predetermined level, and adding the frequency component 
5 multiplied by a constant to the expanded image, or 

subtracting the frequency component multiplied by a constant 
from the expanded image. This configuration is preferable 
because the contrast can be maintained at a predetermined 
level, even when, depending on how the rank order processing 
10 is performed, the contrast in the original image can not be 
"4 maintained. To determine whether the contrast is maintained 
jH at the predetermined level, the contrast of the foreground 
vj3 and the background of the original image is compared with 
l i\ the contrast of those sides of the expanded image, and the 

>i§5 result is compared with a predetermined threshold value. 

gi According to a third aspect of the invention, an image 
jr! transform method comprises the steps of: reading a target 
p pixel and neighboring pixels in original image data; 
|SB? employing the target pixel and the neighboring pixels to 
20 calculate directional differences for the horizontal, the 

vertical, the right oblique and the left oblique directions; 
employing the directional differences to determine a strong 
correlated direction; and performing linear interpolation 
for the target pixel using the neighboring pixels arranged 
25 in the strong correlated direction* 

The image transform method further comprises the steps of: 
reading peripheral pixels arranged within a predetermined 
mask range adjacent to the target pixel and/or the 
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neighboring pixels; and accumulating differences between the 
peripheral pixels, and between the target pixel and the 
neighboring pixels; and determining an interpolation 
direction, based on the cumulative value of the differences, 
5 and performing interpolation in the interpolation direction. 
As a result, even for thin lines obtained by anti-aliasing 
(process for fusing the dots at borders to the background) , 
the direction of the foreground can be obtained 
substantially correctly. 

jlo According to a fourth aspect of the invention, an image 
\M transform method comprises: an input step of entering 
ifl original image data to be expanded by a magnification of two 
|f! or more; a first process step of reducing the step-shapes or 
:Q chain-shapes of oblique lines appearing when the original 
115 image data are expanded by doubled or greater in size; a 
gi second process step of expanding, in the oblique direction, 
jl! the structure of the original image data, and reducing a 
□ bulging shape appearing when a portion is expanded whereat 
^ vertical and horizontal lines of the original image data 
20 cross each other; and an output step of outputting an image 

expanded by the magnification of two or more using results 

of the first and second process steps. 

Further, to achieve the above objects, according to a fifth 
aspect of the present invention, an image processing 
25 apparatus comprises: input means for entering original image 
data to be expanded; vertical and horizontal directional 
interpolation means for interpolating the original image 
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data in the vertical and horizontal directions; vertical and 
horizontal directional correlation reduction means for 
reducing correlation of the obtained image in the vertical 
and horizontal directions; oblique direction detection means 
5 for detecting an oblique direction having a strong 

correlation with a target pixel and neighboring pixels in 
the original image data; and directional interpolation means 
for employing the neighboring pixels in the detected oblique 
direction to perform interpolation in the oblique direction. 

So The image processing apparatus further comprises: generation 
]^ means for generating expanded image data based on an image 
to obtained by the vertical and horizontal directional 
!f! correlation reduction means and an image obtained by the 
=fj oblique directional interpolation means. This configuration 
115 is preferable because the advantages offered by these means 
gi facilitate the output of an image the user preference, for 
!r! example. 

r ^ The image processing apparatus further comprises: input 
means for entering, as an adjustment value, the personal 

20 preference of a user concerning image quality, wherein the 
generation means employs the adjustment value to synthesize 
the image obtained by the vertical and horizontal 
directional correlation reduction means with the image 
obtained by the oblique directional interpolation means. 

25 The adjustment value entered at the input means need not be 
the one that is directly used by the image processing 
apparatus, and a sensuous input value (e.g., setup due to 
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comparison, such as softer or clearer, or a gradual setup) 
relative to the personal preference of the user may be 
transformed for synthesization. 

The vertical and horizontal directional correlation 
5 reduction means performs the ranked median value selection, 
such as the median operation, for the target pixel and the 
neighboring pixels in the linearly expanded image data, and 
thereby reduces the correlation of an image in the vertical 
and horizontal direction. This configuration is superior 

[|0 because a high quality, expanded image can be obtained by 

l fi using a small, simple processing apparatus. 

);! Further, the oblique direction detection means employs 

:J3 differences between the target pixel and the neighboring 

;L, pixels to detect, with strong correlation, the oblique 

IJ15 direction, and the oblique directional interpolation means 

jri performs linear interpolation in the oblique direction 

O detected by the oblique direction detection means. 

According to a sixth aspect of the invention, an image 
processing apparatus comprises: a vertical and horizontal 

20 linear interpolation unit for forming an image by linearly 

expanding original image data in the vertical and horizontal 
directions; and a vertical and horizontal directional 
correlation reduction processing unit for reducing, for the 
image, a vertical and horizontal directional correlation 

25 using a rank order processing, and for generating an 
expanded image . 
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According to a seventh aspect of the invention, an image 
processing apparatus comprises: an interpolation direction 
determination unit for reading a target pixel and 
neighboring pixels in original image data, for calculating 
directional differences between the target pixel and the 
neighboring pixels for vertical, horizontal, right oblique 
and left oblique directions, and for determining an 
interpolation direction based on the directional 
differences; and an oblique directional linear interpolation 
unit for performing linear interpolation for the target 
pixel by using the neighboring pixels arranged in the 
determined interpolation direction. The interpolation 
direction determination unit reads peripheral pixels 
arranged within a predetermined mask range adjacent to the 
target pixel and/or the neighbor pixels and adds together 
the differences between the peripheral pixels and the target 
pixel and the neighbor pixels, and determines the 
interpolation direction based on the cumulative value of the 
differences. This configuration is preferable because, when 
the interpolation direction can not be identified using only 
the directional differences among the directions, the 
interpolation direction can be accurately determined. 

According to an eighth aspect of the present invention, an 
image display device, for transforming low-resolution 
original color image data into high-resolution expanded 
color image data, and for outputting the high-resolution 
expanded color image data, comprises: first image expansion 
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means for outputting an expanded image wherein the vertical 
and horizontal structure is maintained with reducing the 
step-shapes or chain-shapes of oblique lines in the linearly 
expanded image; second image expansion means for expanding 
5 the structure of the original color image data in the 

oblique direction, for reducing a bulging shape that appears 
at intersections of lines, and for outputting an expanded 
image; and display means for employing the expanded images 
obtained by the first and the second image expansion means 
JO to display a final image. 

M z The original color image data includes thin lines obtained 

0 by anti-aliasing, and the second image expansion means 

i\ performs interpolation, while correctly identifying the 
fl foreground based on pixels constituting the original thin 
Li 5 lines, not based on pixels obtained by anti-aliasing. 

Z Brief Description of the Drawings: 

Fig. 1 is a diagram showing an image processing apparatus 
according to one embodiment of the present invention. 
Fig. 2 is a flowchart for explaining the processing 
20 performed by the vertical and horizontal interpolation unit 
12 . 

Figs. 3A and 3B are diagrams for explaining the reduction of 
a step-shape of boundary using the rank order processing. 
Fig. 4 is a flowchart for explaining the processing 
25 performed by the oblique interpolation unit 15. 

Figs. 5A, 5B and 5C are diagrams showing example pixel 
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patterns . 

Figs. 6D, 6E, 6F, 6G and 6H are diagrams showing example 
pixel patterns. 

Fig. 7 is a flowchart for explaining directional search 
processing using a cumulative difference. 

Figs. 8A and 8B are diagrams showing example pixel patterns 

for which a cumulative difference is employed. 

Fig. 9 is a flowchart for explaining the directional search 

processing using a doubled difference. 

Figs. 10A, 10B and IOC are diagrams for explaining an 

oblique line before expansion and a final image obtained 

using the conventional expansion method. 

Figs. 11A and 11B are diagrams for explaining a final image 
obtained for the embodiment . 

Figs. 12A and 12B are diagrams for explaining a crossed 
portion before expansion and a final image obtained using 
the conventional expansion method. 

Fig. 13 is a diagram for explaining a final image obtained 
for the embodiment . 

Figs. 14A and 14B are diagrams showing the state wherein an 
isolated point is expanded. 

Description of the Symbols 

11: Original image data input unit 

12: Vertical and horizontal interpolation unit 

13 : Linear interpolation performed in the vertical and 

horizontal direction order 

14: Vertical and horizontal directional correlation 
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reduction process 
15: Oblique interpolation unit 
16: Interpolation direction determination 
17: Oblique linear interpolation 
5 18: Final result generator 
19: Final image output unit 

Detailed Description of the Invention: 
sjp Preferred Embodiment 

\Q The preferred embodiment of the present invention will now 
zlO be described in detail while referring to the accompanying 
iJ3 drawings . 

01 An image processing apparatus according to this embodiment 
;Z is, for example, a liquid crystal display monitor, which 

O receives, from a connected host apparatus, such as a PC, 
H5 original image data consisting of R (red) , G (green) and B 
(blue) color signals for a text screen or a computer aided 
design (CAD) screen, and performs a resolution transform for 
the image data, after which it outputs the results on a 
display unit, such as a liquid crystal display cell. For an 
20 LCD monitor or a notebook PC, this liquid crystal display 

monitor can be implemented, for example, by an internal LCD 
controller. The image processing apparatus is effective 
when, for example, original color image data generated by a 
low-resolution VGA (Video Graphics Array) (640 X 480 dots) 
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graphic chip is displayed on a high-resolution UXGA (Ultra 
Extended Graphics Array) (1600 X 1200 dots) LCD display. 
Further, for a notebook PC the image processing apparatus is 
implemented to include the host side. It should be noted 
that no further detailed explanation will be given for a 
liquid crystal display cell and the X and Y drivers that 
supply power to the cell. 

In this embodiment, to improve the conventional method 
whereby one pixel in an original image is expanded as a 
rectangular group of pixels, and the oblique lines of the 
original image assume a step shape or a chain shape in the 
resultant image, a method and an apparatus are proposed for 
generating a clear, expanded image wherein the structural 
balance of the characters or the graphic design of the 
original image is maintained. The factors for determining 
the adequateness of an expansion are those that are 
considered to be effective, so that a user feels natural to 
an expanded image when he or she looks at a graphic design, 
such as a text screen or a computer aided design screen, 
displayed on a high-definition LCD. It should be noted that 
this method can also be applied when a user views a natural 
image, and is not limited in use only to an expanded image 
displayed on an LCD screen. 

Fig. 1 is a diagram illustrating an image processing 
apparatus according to the embodiment of the present 
invention. The image processing apparatus roughly 
comprises: an original image data input unit 11, a vertical 
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and horizontal interpolation unit 12, an oblique 
interpolation unit 15, a final result generator 18 and a 
final image output unit 19. The original image data input 
unit 11 receives original image data during the process 
5 performed for each coordinate, and pixel values are supplied 
thereto in a time-series manner as, for example, a 
one-dimensional sequential data stream that is received as a 
horizontal scan line. The vertical and horizontal 
interpolation unit 12 performs in the named order, for the 
JO original image data received from the original image data 
■5 input unit 11, a linear interpolation 13 process in the 
| B vertical and horizontal directions, and a vertical and 
;fi horizontal directional correlation reduction process 14 for 
Zl reducing the vertical and horizontal directional 
dl5 correlation. Thus, a large, step-shape portion, acquired as 
L a result of the linear interpolation 13 expansion process, 

iji is changed, and a smooth shape is produced by the vertical 
\Z and horizontal directional correlation reduction process 14 
□ and is subsequently output. The oblique interpolation unit 
rj 2Q 15 performs, for the original image data received from the 
original image data input unit 11, a process 16 for 
determining the interpolation direction and an oblique 
linear interpolation 17. Thus, the oblique direction can be 
accurately determined and the interpolation process can be 
25 performed in the oblique direction. 

The final result generator 18 synthesizes, at a 
predetermined ratio, the results received from the vertical 
and horizontal interpolation unit 12 and the oblique 
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interpolation unit 15 in order to maintain the character or 
graphic design balance. The predetermined ratio for this 
synthesization can be obtained as an adjustment value for 
image quality that reflects the personal preference of a 
5 user. Further, this adjustment value can be obtained, for 
example, by a user employing input means (not shown) of a PC 
to enter data representing his or her reaction to the 
display of an expanded image. The final image output unit 
19 outputs the results obtained from the final result 
10 generator 18 to a liquid crystal display cell driver, for 

p example, which displays the image obtained by this 

M embodiment on a liquid crystal display. 

f! When a low-resolution PC screen image is to be expanded and 

p displayed on a high-definition LCD, according to many 

:J5 expansion methods, an oblique one pixel-width line in the 

jl original image is transformed into a step-shape or 

l! chain-shape line consisting of a rectangular group of 

!Tj pixels. As the main factor in this case, it is assumed that 

for the calculation dependence is to be placed on the 
20 sequential arrangement of the interpolated pixels, and a 
strong correlation, such as that in the vertical and 
horizontal directions, occurs among interpolated pixels. 
Whereas, if the calculation is performed by using peripheral 
pixels arranged in the direction in which the step-shape or 
25 chain-shape line appears, this problem may be avoided. 
However, at the same time, the vertical and horizontal 
correlation is reduced, and at the portion where the 
vertical and horizontal lines of a font cross each other, an 
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oblique crossing that differs from the original image 
appears, so that the balance of the font is distorted, or at 
the point at which the lines cross a large round dot will 
appear. Further, if an isolated point on an original image 
5 is interpolated with weak correlation in the vertical and 
the horizontal directions, the isolated point will be 
regarded as a crossing point for oblique lines, and will be 
expanded in an oblique direction, so that it is faded and 
blurred. In this embodiment, in order to avoid these 
JO phenomena, the final result generator 18 generates a final 
n expanded image using the vertical and horizontal 
^ interpolation results, which are output by the vertical and 
ri horizontal interpolation unit 12 in order to maintain design 
i\ balance, and the oblique interpolation results, which are 
JJ5 output by the oblique interpolation unit 15 in order to 
L, reduce the step-shapes or chain-shapes of oblique lines. 

Z Fig. 2 is a flowchart for explaining the processing 

□ performed by the vertical and horizontal interpolation unit 
12. First, vertical and horizontal bilinear interpolation 

20 is performed for the original image data received from the 
original image data input unit 11 (step 101) . The vertical 
and horizontal linear interpolation 13 described while 
referring to Fig. 1 can be performed using the conventional 
bilinear interpolation method, and cubic convolutional 

25 interpolation or interpolation using a B-spline can also be 
performed, so that the interpolation method is not 
especially limited. Following this, as the vertical and 
horizontal directional correlation reduction process 14, a 
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target pixel and peripheral pixels are read (step 102) and 
sorted (step 103), and the median value is determined as the 
pixel value at targeted coordinates (step 104) . 
Specifically, the peripheral pixels that are included in a 
5 mask of an integer matrix that does not exceed the expansion 
ratio (e.g., a mask of two rows and two columns if the 
expansion ratio is 2.5) are sorted in order in accordance 
with value, and the median value is output as the process 
result. As the process for outputting the median value, a 
.JO rank order processing is used for outputting a predetermined 
2 order value, or a process for repetitively copying the 

pixels in the mask and sorting them, and for performing a 
:J3 rank order processing for the sorted results can be 
If! employed. So long as the vertical and horizontal 
:|15 directional correlation can be reduced, any acceptable 
m arbitrary process can be employed. Then, whether the 

ill current pixel is the last one is determined (step 105) . If 

]Z the pixel is not the last one, program control returns to 
O step 102 to repeat the above processing. If the current 

p !20 pixel is the last one, program control advances to step 106. 

At step 106, a check is performed to determine whether the 
contrast of the original image can be maintained through the 
above processing. For this determination, the difference in 
the luminances at a boundary portion is obtained and is 
25 compared with a threshold value. When the luminance 
difference is smaller than the threshold value, it is 
ascertained that the contrast can not be maintained. Whether 
or not the enhancement process is required depends on the 
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employment of the rank order, and when the contrast of the 
original image can be maintained, the pixel value determined 
at step 104 is output unchanged to the final result 
generator 18. But when it is determined the contrast of the 
5 original image can not be maintained, the image enhancement 
process is performed at steps 107 to 109. The process 
performed at steps 107 to 109 is an image sharpening process 
called unsharp masking. During this process, vertical and 
horizontal directional correlations are reduced at the 
JO portion whereat the median value is output, and the high 
S frequency component of the image is enhanced in order to 
JS restore the image contrast obtained from the vertical and 
y3 horizontal linear interpolation 13. Specifically, the high 
Zl frequency component is extracted from the image (step 107), 
=£15 and the extracted image is multiplied by a constant value 
;L (step 108) . The pre-extracted image and the multiplied 

g| image are added together (step 109) , and the result obtained 
Ij! at step 109 is output to the final result generator 18. 

r ^ As is described above, the vertical and horizontal 
20 interpolation unit 12 can linearly expand original image 
data in the vertical and horizontal directions, or vice 
versa, and can thereafter perform a non-linear process for 
changing the vertical and horizontal directional image that 
is obtained to a set of non-directional correlation pixels. 
25 With this configuration, a non-linearly expanded image, 

wherein the step-shapes or chain-shapes of oblique lines are 
reduced, can be output without distorting the balance of the 
linear structure forming a character or a graphic in the 
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original image, and without losing an isolated point in the 
image . 

Figs. 3A and 3B are diagrams for explaining the reduction of 
a step-shape of boundary using the above described rank 
5 order processing. In Fig. 3A is shown the state obtained 
through the expansion process for which the vertical and 
horizontal linear interpolation 13 in Fig. 1 is used, and in 
Fig. 3B is shown the state obtained when the vertical and 
horizontal directional correlation reduction process 14 in 

|10 Fig. 1 is used. In Figs. 3A and 3B, the pixel values in 49, 
7x7 windows are represented as expansion results, with 

13 "32", for example, being defined as bright and "0" being 

defined as dark. A thick solid line indicates a boundary 

0 line, and as is indicated by the thick solid line in Fig. 

'45 3A, a boundary having large steps occurs as the result of 

fl the expansion process for which only the vertical and 

%i horizontal linear interpolation 13 is used. 

In this embodiment, the rank order processing is performed 
after the expansion process in Fig. 3A for which the 

20 vertical and horizontal linear interpolation 13 is used. 
Specifically, the rank order processing is performed by 
raster-scanning a window, enclosed by a broken line 21 in 
Fig. 3A, in the direction indicated by an arrow 22, and the 
pixel positions are adjusted in accordance with the even 

25 numbers of the pixels in the window. In the area enclosed 
by the broken line 21, four windows are defined by a pixel 
23 at the second row and the second column, a pre-scan pixel 
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(second row and first column) and the pixels along the 
preceding line (first row and first column and second 
column), and the median operation, for example, is performed 
for the four windows to determine the output value for the 
5 pixel 23 on the second row in the second column. The 

results obtained by the rank order processing are shown in 
Fig. 3B. The " *" symbols along the first row and along the 
first column represent "Don't care", because the values of 
the pixels before the scanning and along the preceding row 
,40 are unknown. 

r; The pixel, for example, on the second row in the sixth 
03 column in Fig. 3A is "0". When the median operation is 
jSj performed for this pixel, the pre-scan pixel (second row and 
B fifth column) is "32" and the pixels along the preceding row 
U5 are "32" and "0" . When these pixels are rearranged in the 
01 ascending order, "0, 0, 32, 32", the median value is "16". 
U~i As a result, as is shown in Fig. 3b, the pixel on the second 

□ row in the sixth column is transformed to "16". Similarly, 
the pixel, for example, on the third row in the fourth 
20 column in Fig. 3A is "0", and the vertical and horizontal 
pixel values in the window are rearranged in the ascending 
order, "0, 32, 32, 32". Therefore, the median value is 
"32", and the output value is indicated on the third row in 
the fourth column in Fig. 3B. When the rank order 
25 processing has been performed in this manner, it is apparent 
that, as indicated by the thick solid line in Fig. 3B, the 
large steps are removed from the boundary, and further, the 
boundary is interpolated with the median values. As a 
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result, a clear and smooth expanded image can be obtained at 
the current resolution. 

The processing performed by the vertical and horizontal 
interpolation unit 12 will now be described by using a 
general calculation example. 

In this example, a calculation will be described for the 
processing including bilinear interpolation, the use of a 
median filter, and the unsharp masking for an image 
expansion of 2.5 times. 

First, by using the following equation, the weighted mean 
using the distance of interpolation value B k [X] is performed 
for real number coordinates X between two adjacent pixels 
S[X k ] and S[X k+i ] in an original image. 

[Equation 1] 

B k [X] = sS[X k ] + (1 - s)S[X k+1 ], 
for X k ^ X ^ X k+ i, s = X - X k 

Bilinear interpolation is the method used to perform this 
calculation for the order in the vertical and horizontal 
directions . 

When the expanded result obtained by bilinear interpolation 
is defined as B(x, y) , and the size of the mask is 2 X 2, as 
previously described, the median filter employs the 
following equation for the processing. 
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[Equation 2] 

M(x, y) = median{ [B(x-1, y-1) , B(x-1, y) , B(x, y-1), 
B(x, y)]} 

In this example, when the number of vector elements of the 
target is odd, the median value for output is provided by 
median operation, or when the number of elements is even, 
the output value is determined in the next process. For the 
above equation, when the pixel values are rearranged in the 
ascending order, as [Ri, R 2 , Rs, Rd (Ri ^ R 2 ^ Rs ^ Ra) , the 
output provided by median operation is 

[Equation 3] 

M(x, y) - (R 2 + R 3 )/2 

When the output of an integer value is expected, a floor 
calculation for rounding off the calculation results in the 
negative infinite direction is performed, and the result 

[Equation 4] 

Lm(x, y)J 

is employed as the output value. Further, for the 
extraction of a high frequency component, simple means can 
be employed, such as a second-order Volterra filter in the 
following equation proposed in the IEEE Transactions on 
Image Processing, Vol, 5, No. 6, pp. 950-963, June 1996. 

[Equation 5] 
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V(x, y) = 2M 2 (x, y) - M(x, y-l)*M(x, y+1) 
- M(x-1, y)»M(x+l, y) 

In the same manner as in a method frequently used for 
unsharp masking, a specific scalar value y is provided to 
5 obtain the final results using the following equation. 

[Equation 6] 

N(x, y) = M(x, y) + y # V(x, y) 

1 In this case, instead of a Volterra filter, a Laplacian 

^ filter may be employed for a high-pass filter, and the image 

JIO enhancement method is not thereby limited. 

As is described above, when the thus arranged vertical and 
ff horizontal interpolation unit 12 in Fig. 1 is employed, a 

step-shape or chain-shape of an oblique line can be reduced, 
if while output with no distortion is maintained in the 
45 vertical and horizontal crossed lines of a character that is 
obtained using common linear interpolation. However, since 
this process is primarily for the rearrangement of the 
pixels in order to reduce the vertical and horizontal 
correlation, a problem of uneven coloration arises. Thus, 
20 in this embodiment, to resolve this problem using the 
interpolation results wherein the step-shapes or 
chain-shapes of oblique lines and uneven coloration appear 
less frequently, the oblique interpolation unit 15 in Fig. 1 
is provided, even though the vertically and horizontally 
25 crossed lines are distorted. 
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Fig. 4 is a flowchart for explaining the processing 
performed by the oblique interpolation unit 15. In this 
embodiment, basically, the interpolation direction 
determination process 16 is performed, and then the oblique 
linear interpolation process 17 is performed, wherein pixels 
in the interpolation direction are employed for linear 
interpolation . 

First, the neighboring pixels at an inverse projected point 
of a target expansion point are obtained from the original 
image data input unit 11 (step 201), and differences for the 
horizontal and the vertical, and the right and the left 
oblique directions are calculated (step 202) . Specifically, 
when inverse projection is performed from the target 
coordinates obtained by expansion to the original 
coordinates, four peripheral pixels are employed to 
calculate differences in the horizontal and the vertical, 
and the right and the left oblique directions. Then, the 
differences are compared with each other, i.e., a check is 
performed to determine whether the horizontal or vertical 
difference is equal to or smaller than a threshold value 
(step 203) . 

When the horizontal or the vertical difference is equal to 
or smaller than the threshold value, a check is performed to 
determine whether the difference in the left oblique 
direction is equal to or smaller than the threshold value 
(step 204) . And if the difference in the left oblique 
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direction is equal to or smaller than the threshold value, a 
check is performed to determine whether the difference in 
the right oblique direction is equal to or smaller than the 
threshold value (step 205) . Then, should the difference be 
equal to or smaller than the threshold value, i.e., when the 
difference in each of the two oblique directions is equal to 
or smaller than the threshold value, the vertical and 
horizontal directions are determined to be interpolation 
directions (step 210) . Whereas if at step 205 the 
difference in the right oblique direction is greater than 
the threshold value, the left oblique direction is 
determined to be the interpolation direction (step 211) . If 
at step 204 the left oblique direction is found to be 
greater than the threshold value, a check is performed to 
determine whether the difference in the right oblique 
direction is equal to or smaller than the threshold value 
(step 206) . In this case, if the difference in the right 
oblique direction is equal to or smaller than the threshold 
value, the right oblique direction is determined to be the 
interpolation direction (step 212) . But if at step 206 the 
difference in the right oblique direction is found to be 
greater than the threshold value, then the vertical and 
horizontal directions are determined to be interpolation 
directions (step 213). 

If at step 203 the difference in the horizontal or vertical 
direction is greater than the threshold value, a check is 
performed to determine whether the difference in the left 
oblique direction is equal to or smaller than the threshold 
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value (step 207). Then, when that difference is equal to or 
smaller than the threshold value, a check is performed to 
determine whether the right oblique difference is equal to 
or smaller than the threshold value (step 208). And if the 
difference is equal to or smaller than the threshold value, 
i.e., when the difference in each of the two oblique 
directions is equal to or smaller than the threshold value, 
the directional search is performed using the cumulative 
value of the differences (step 214) . But when at step 208 
the difference in the right oblique direction is greater 
than the threshold value, the left oblique direction is 
determined to be the interpolation direction (step 215) . 
Again, if at step 207 the difference in the left oblique 
direction is greater than the threshold value, a check is 
then performed to determine whether the difference in the 
right oblique direction is equal to or smaller than the 
threshold value (step 2 09) . And when the difference in the 
right oblique direction is equal to or smaller than the 
threshold value, the right oblique direction is determined 
to be the interpolation direction (step 216) . If, however, 
at step 209 the difference in the right oblique direction is 
greater than the threshold value, the directional search is 
performed using a doubled difference (step 217) . It should 
be noted that the threshold value used throughout this 
process is an arbitrary value, determined while 
consideration is given to the states of the pixels, and is 
stored in a memory (not shown) . 

Finally, based on a determination made at one of steps 210 
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to 217, linear interpolation is performed using pixels 
arranged in the interpolation direction (step 218) . 
Thereafter, the results obtained by linear interpolation are 
output to the final result generator 18. 

5 Figs. 5A to 5C and 6D to 6H are diagrams showing example 
pixel patterns that correspond to the respective cases 
described in Fig. 4. In these diagrams, the relationship 
between the pixel values is conceptually represented, whose 
relationship consists of an inverse projected point of a 
SO target expansion point and the neighboring pixels of the 
r! projected point, and in this embodiment, bilinear 
ffl interpolation is performed using these four neighboring 
%l pixels. The pixel pattern in Fig. 5A corresponds to step 
?fl 210 in Fig. 4, the pixel pattern in Fig. 5B corresponds to 

p45 step 211, and the pixel pattern in Fig. 5C corresponds to 
yl step 212. Further, the pixel pattern in Fig. 6D corresponds 

to step 213, the pixel pattern in Fig. 6E corresponds to 
□ step 215, the pixel pattern in Fig. 6F corresponds to step 

s 216, the pixel pattern in Fig. 6G corresponds to step 214, 
20 and the pixel pattern in Fig. 6H corresponds to step 217. 

In these pixel patterns, the circles represent pixels, and 
when the pixel values are near each other, i.e., when the 
difference between them is equal to or smaller than a 
threshold value, the pixels are connected by thick, solid 
25 lines. 

According to the determination at step 210 in Fig. 4, the 
pixel pattern in Fig. 5A is that found when the differences 
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in all the horizontal and vertical, and right and left 
oblique directions are equal to or smaller than the 
threshold value and are near each other. In this case, 
oblique interpolation need not be performed, and 
5 conventional interpolation is performed in the vertical and 
horizontal directions . 

When the difference in the vertical or horizontal direction 
is equal to or smaller than the threshold value, and when 
^ the difference in one of the oblique directions is also 
-:J10 equal to or smaller than the threshold value, the pertinent 
■ft oblique direction is determined to be the interpolation 
J3 direction. In the other cases, the vertical and horizontal 
j^ij directions are determined to be the interpolation directions 
B (steps 211, 212 and 213), The pixel pattern when the left 

p|5 oblique direction is determined to be the interpolation 
01 direction is shown in Fig. 5B; the pixel pattern when the 
right oblique direction is so determined is shown in Fig. 
O 5C; and the pixel patterns in the other cases when the 

vertical and horizontal directions are so determined are 
20 shown in Fig. 6D. When the difference in the horizontal or 
vertical direction is greater than the threshold value, the 
differences in the oblique directions are compared as shown 
in the pixel patterns in Figs. 6E and 6F, and the direction 
having a value smaller than the threshold value is 
25 determined to be the interpolation direction (steps 215 and 
216) . 

At this time, the following two cases must also be coped 
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with. In the first case, as is shown in Fig. 6G, the 
difference in the horizontal and vertical directions are 
equal to or greater than the threshold value, and the 
differences in the right and left oblique directions are 
5 equal to or smaller than the threshold value. Here, the 

differences between peripheral pixels that are adjacent to 
the diagonal lines connecting the above four neighboring 
pixels are added together, and the cumulative value is 
employed to determine the interpolation direction (step 214 
40 in Fig. 4) . 

" r ; Fig. 7 is a flowchart for explaining a directional search 

0 performed using the cumulative value of differences. First, 

t\ the pixels in a 6 X 6 mask around the inverse projected 

fl point are read (step 3 01) . The difference accumulation 

45 range is so defined that it is large enough to cover the 

P essential structure, in the original image, of a font 

ij segment of one pixel line. In many cases, a 6 X 6 mask 

-J range can provide a sufficient window size that has two more 

peripheries than a 2 X 2 window. Here it should be noted, 
20 however, that no limit is imposed on the ranges that can be 

used for masks . 

The cumulative value DL of the pixel difference values for 
the left oblique direction is obtained (step 302) , as is the 
cumulative value DR of the pixel difference values for the 
25 right oblique direction is obtained (step 303). Thereafter, 
a check is performed to determine whether the absolute value 
for (DL - DR) is equal to or smaller than a threshold value 
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(step 3 04) . When the absolute value is equal to or smaller 
than the threshold value, the vertical and the horizontal 
directions are determined to be the interpolation directions 
(step 305) . When at step 304 the absolute value for (DL - 
DR) is greater than the threshold value, the cumulative 
values DL and DR are compared (step 3 06) . If the cumulative 
value DL, for the pixel difference values in the left 
oblique direction, is greater, the left direction is 
determined to be the interpolation direction (step 3 07) . 
Whereas when the cumulative value DR, for the pixel 
difference values in the right oblique direction, is 
greater, the right direction is determined to be the 
interpolation direction (step 308) . In this manner, the 
side having the smaller cumulative value is determined to be 
the background, and the opposite diagonal direction is 
identified as the interpolation direction. When the 
difference between the cumulative value for the right 
oblique direction and the cumulative value for the left 
oblique direction is equal to or smaller than a threshold 
value, the vertical and horizontal directions are determined 
to be the interpolation directions. 

Figs. 8A and 8B are diagrams showing an example pixel 
pattern for which the cumulative value of the differences is 
employed. The pixel pattern of a portion of the 
alphabetical character "a", enclosed by a thin line in Fig. 
8A, is shown in Fig. 8B. As is shown in Fig. 8B, the 
crossing of the foreground and background in a small font 
has a line width of one dot. When the dot dimensions 6x6 
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are used to view such a font, it is found that the number of 
pixels used for the foreground of the font is smaller than 
the number of pixels used for the background. Therefore, the 
simplest way to identify the foreground is to compare the 
5 number of like colored pixels. However, for thin lines 

obtained, for example, by anti-aliasing (the process used 
for fusing the dots at the boundary with the background) , if 
the thin lines cross, due to the anti-aliasing, the color of 
the pixels at the boundary is mixed with the color used on 
10 the background, and the number of pixels having mixed colors 
5 is smaller than the number of originally colored pixels on 
}3 the foreground. In this embodiment, therefore, for each 
S color, color differences are accumulated for an entire area 
!*j within a dimensionally defined space. Thus, since the color 
;JJ5 of pixels obtained by anti-aliasing is closer to the color 
L of the background, the accumulation difference decreases, 
m and the color used for the foreground is correctly detected 
^ as the color having the largest accumulation difference. 

pr An explanation will now be given for the second case that 

20 must be coped with. In the second case, as is shown by the 
pixel pattern in Fig. 6H, each of the differences in the 
horizontal and vertical directions and in the right and left 
oblique directions are greater than the threshold value 
(step 217 in Fig. 4) . In this case, a directional search 
25 using a doubled difference is performed. 

Fig. 9 is a flowchart for explaining a directional search 
performed using a doubled difference. For this process, 
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first, a horizontal difference h, a vertical difference v, a 
left oblique difference dl and a right oblique difference dr 
are read (step 401) . Then, the process is branched, 
depending on whether the absolute value of a difference 
5 between the oblique differences is equal to or smaller than 
the threshold value, i.e., a check is performed to determine 
whether the absolute value of (dl - dr) is equal to or 
smaller than the threshold value (step 402) . If the 
absolute value is equal to or smaller than the threshold 
JO value, the interpolation direction is determined to be the 
IB vertical and the horizontal directions (step 403) . But if 
R the absolute value is greater than the threshold value, the 
03 differences in the right and left oblique directions are 
:f : compared (step 404) . 

!Lj5 When at step 404 dl < dr, i.e., when the difference dl in 
01 the left oblique direction is smaller, a check is performed 
;«* to determine whether h < dl or v < dl is satisfied between 
Q the left oblique difference and either the horizontal 

difference or the vertical difference (step 405) . When h < 
20 dl or v < dl, the interpolation direction is determined to 
be the vertical and horizontal directions (step 407). When 
h < dl or v < dl is not satisfied, i.e., the difference in 
the left oblique direction is smaller, the left oblique 
direction is determined to be the interpolation direction 
25 (step 408) . 

Similarly, when dl < dr is not satisfied at step 404, i.e., 
when the difference dr in the right oblique direction is 
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smaller, a check is performed to determine whether h < dr or 
v < dr is satisfied between the right oblique difference and 
the horizontal difference or the vertical difference (step 
406) . When h < dr or v < dr, the interpolation direction is 
5 determined to be vertical and the horizontal directions 

(step 409). But if h < dr or v < dr is not satisfied, i.e., 
when the right oblique difference is smaller, the right 
oblique direction is determined to be the interpolation 
direction (step 410) . 



y§0 Thus, the same linear interpolation as the conventional 

;^ bilinear interpolation is performed by using four 

=fl neighboring pixels arranged in the thus determined 

%\ interpolation direction and at the inverse projected point. 

'43 It should be noted that when the pixels in the oblique 

5U5 direction are employed, the distance in the oblique 

ffl direction can be employed as the distance between the 

original image pixel and the interpolation pixel used for 
D bilinear interpolation. Further, when the distances along 
1 the vertical and horizontal directions are employed, because 
20 of the similarity in the relationship established for the 
distance ratio, substantially the same results can be 
obtained as when an oblique direction is employed. Note 
that the number of directional divisions used in the above 
explanation is only an example, and that another number may 
25 be employed. 

As is described above, since, in this embodiment, unlike in 
USP 5,991,463, the oblique interpolation unit 15 determines 
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the direction using the cumulative difference values, it can 
be expected that reliability will be statistically improved 
and that the number of erroneous determinations will be 
reduced. As is understood from the pixel selection method, 
5 the process performed by the oblique interpolation unit 15 
renders the step-shapes or chain-shapes of oblique lines 
substantially not remarkable. Further, since the reduction 
of the correlation, which is caused by the vertical and 
horizontal directional correlation reduction process 
40 performed by the vertical and horizontal interpolation unit 
5 12, does not occur, uneven coloration is not remarkable in 
l M s an output image expanded by a multiple of two or more. 
;fl However, according to this method, the pixels in the oblique 
:f: directions are employed for a Chinese (Kanji) character 

;£15 wherein many lines cross vertically and horizontally. Thus, 
*L the balance of the image obtained by expanding a character 

g! or graphic included in the original image are distorted; a 

large round dot (a bulging shape) , for example, appears at 
D crossing points in the original image; and following the 

5 ~20 image expansion, an isolated point is small and blurred. 

Therefor, in this embodiment, the non-linear interpolation 
result obtained by the vertical and horizontal interpolation 
unit 12, and the result obtained by the oblique 
interpolation unit 15 are employed together, so that the 
25 disadvantages of the two offset each other and their 

advantages are established. The main advantage of the 
non-linear interpolation result obtained by the vertical and 
horizontal interpolation unit 12 is that the balance of 
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characters and graphics is maintained. The advantage of the 
interpolation result obtained by the oblique interpolation 
unit 15 is a reduction in step-shapes or chain-shapes of 
oblique lines and in uneven coloration. As an example of 
5 the generation of an image for which these two advantages 
are provided, when an image obtained using non-linear 
interpolation is represented by N(x,y) and an image obtained 
using oblique interpolation is represented by S(x,y), a 
final image R(x,y) can be obtained by using the adjustment 
40 value a. 

y [Equation 7] 

5 R(x,y) = o£N(x,y) + (1 - a)S(x,y) 

Since the image for which the two advantages are provided is 
3 generated by using the adjustment value a, it is possible to 

ld5 obtain a clear final image wherein a step-shape or 
jjf chain-shape of oblique line or uneven coloration in the same 

i color area is not noticeable, and the balance of the lines 

composing a character or graphics in an original image is 
not distorted. As is described above, the adjustment value 
20 a can be determined from an entry made by a consonant with 
his or her personal preference. Further, the user entry 
need not be the value a, and based on the predetermined 
value set by the user, the adjustment value may be 
transformed internally by the image processing apparatus. 
25 As an example setup by a user, a sensuous expression, such 
as softness or clarity, can be designated using a number of 
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levels by employing a display or a pointing device. 

Figs. 10 to 14 are diagrams for explaining the final image 
in this embodiment when an expansion of 2.5 times is 
employed. An oblique line before expansion is shown in Fig. 
5 10A, and the result obtained by the expansion of the oblique 
line in Fig. 10A using common bilinear interpolation is 
shown in Fig. 10B. Next, in Fig. 10C, the result is shown 
that is obtained by interpolating the oblique line in Fig. 
10A using a third-order B-spline. The common bilinear 
3o interpolation shown in Fig. 10B is the expansion method 

currently equipped in many LCD monitors, and it is apparent 
O that when this method is used to expand an image, the 
il step-shape or chain-shape of oblique line is remarkable. 
13 Since the interpolation for which the third-order B-spline 

45 in Fig. 10C is used, is regarded as a method for obtaining a 
Ji precision near the theoretical upper limit that can be 
7* achieved by linear interpolation, this method is shown for 

3 comparison. It is apparent, however, that even when 

interpolation using a third-order B-spline is employed, 
20 step-shapes or chain-shapes of oblique lines are remarkable. 

Fig. 11A is a diagram showing the expanded state obtained, 
by the vertical and horizontal interpolation unit 12 of this 
embodiment, merely through non-linear interpolation. Due to 
the use of non-linear interpolation, the step-shapes or 
25 chain-shapes of oblique lines are reduced; however, changes 
in luminance, the cause of uneven coloration, occur in a few 
places. Fig. 11B is a diagram showing the final image of 
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the oblique line that is output in this embodiment, while 
taking into consideration the result obtained by the oblique 
interpolation unit 15. As is described above, according to 
the embodiment, the number of step-shapes or chain-shapes of 
5 oblique lines, which occur during the expansion of an 

oblique image, is reduced, and an expanded image can be 
obtained wherein uneven coloration is less remarkable. 

Fig. 12A is a diagram before expansion of an example pattern 
in which there is a crossed portion. While Fig. 12B is a 
So diagram showing the pattern of a final image provided by the 
p? oblique interpolation unit 15 of this embodiment. Since the 
y3 oblique interpolation unit 15 determines that the crossed 
»; portion is continued obliquely, as is shown in Fig. 12B, a 
ifl phenomenon occurs whereby a round dot having a bulging shape 
Ll5 is located at the crossed portion. In this embodiment, 
ij! since as is shown in Fig. 13 both the vertical and 

horizontal interpolation unit 12 and the oblique 
O interpolation unit 15 are employed to generate a final 

r ~ result, the above problem can also be resolved. 

20 Figs. 14A and 14B are diagrams showing the state wherein an 
isolated point is expanded. If an isolated point, 
represented in the original image by a single pixel, is 
expanded in the oblique direction by employing the method 
using pixels arranged in the direction, the point is faded 

25 out, as is shown in Fig. 14A. However, since in the 

embodiment non-linear interpolation in the vertical and 
horizontal directions is taken into account, an expanded 
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image can be obtained without undue fading of the isolated 
point occurring. 

As is described above, according to this method, it is 
possible to obtain a final, clear image wherein the number 
5 of step-shapes or chain-shapes of oblique lines is reduced 
and uneven coloration in the same color area is not 
remarkable, and the balance of the lines composing a 
character or graphics in an original image is not distorted. 

As is described above in detail, according to the 
d§0 embodiment, the vertical and horizontal interpolation unit 
l fi 12 is employed, so that the original input image data are 
:J3 linearly expanded in the vertical and horizontal directions, 
jf: or vice versa, and non-linear interpolation is performed for 
:J3 an expanded image having the vertical and horizontal 
Lj5 directional correlations to obtain a set of pixels having no 
ffl directional correlation. Thus, a non-linear expanded image 

can be obtained wherein step-shapes or chain-shapes of 
□ oblique lines are reduced, without distorting the balance of 

r ~ the line structure of characters or graphics in an original 
20 image, or losing an isolated point. 

Furthermore, the oblique interpolation unit 15 is employed, 
so that the target pixel value in the original image data 
and the neighboring pixel values of the target pixel are 
used to detect connection relationships with the target 
25 pixel. Then, based on the obtained results, pixel selection 
is performed and the direction in which pixels are arranged 
is determined to perform linear interpolation. Following 
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this, the expansion process is performed through linear 
interpolation using peripheral pixels arranged in oblique 
directions. Because of this pixel selection method, the 
occurrence of step-shapes or chain-shapes of oblique lines 
5 can be reduced. 

Furthermore, the final result generator 18 generates an 
image, using a predetermined adjustment value a, based on 
the first image obtained by the vertical and horizontal 
fj interpolation unit 12 and the second image obtained by the 
y|0 oblique interpolation unit 15. Thus, a resolution can be 
Ci provided for a preferable transformed image that includes 
!=5 the advantages provided for the first and second images. 

^0 As is described above, according to the present invention, 
□ the occurrence of step-shapes or chain-shapes of oblique 
pl5 lines is reduced, and the distortion or the absence of lines 
m constituting a font or graphics is also reduced, so that a 
clear, expanded image can be output. 

The present invention can be realized in hardware, software, 
or a combination of hardware and software. The present 

20 invention can be realized in a centralized fashion in one 

computer system, or in a distributed fashion where different 
elements are spread across several interconnected computer 
systems. Any kind of computer system - or other apparatus 
adapted for carrying out the methods described herein - is 

25 suitable. A typical combination of hardware and software 

could be a general purpose computer system with a computer 
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program that, when being loaded and executed, controls the 
computer system such that it carries out the methods 
described herein. The present invention can also be embedded 
in a computer program product, which comprises all the 
5 features enabling the implementation of the methods described 
herein, and which - when loaded in a computer system - is 
able to carry out these methods . 

Computer program means or computer program in the present 
context mean any expression, in any language, code or 
|J0 notation, of a set of instructions intended to cause a system 
^ having an information processing capability to perform a 
f| particular function either directly or after conversion to 
i\ another language, code or notation and/or reproduction in a 
Q different material form. 

115 It is noted that the foregoing has outlined some of the more 

I" pertinent objects and embodiments of the present invention. 

3 This invention may be used for many applications. Thus, 

although the description is made for particular arrangements 
and methods, the intent and concept of the invention is 

20 suitable and applicable to other arrangements and 

applications. It will be clear to those skilled in the art 
that other modifications to the disclosed embodiments can be 
effected without departing from the spirit and scope of the 
invention. The described embodiments ought to be construed 
25 to be merely illustrative of some of the more prominent 

features and applications of the invention. Other beneficial 
results can be realized by applying the disclosed invention 
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in a different manner or modifying the invention in ways 
known to those familiar with the art. 
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